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During  any  given  mission,  a  warfighter  faces  demanding  informa¬ 
tion-processing  challenges  while  engaged  in  intense  physical  stresses, 
such  as  heavy  pack  loads,  extreme  temperatures,  unfamiliar  terrain, 
and  high  altitudes.  The  warfighter  must  first  understand  the  mission 
objectives  and  the  information  relevant  for  its  successful  completion 
(e.g.,  the  spatial  layout  of  the  environment,  locations  to  search,  or  the 
appearance  and  estimated  number  of  enemy  combatants  or  weapons). 
This  abstract  information  must  then  be  transferred  to  a  live  and  poten¬ 
tially  hostile  environment,  which  can  have  harsh  and  unknown  condi¬ 
tions.  During  the  course  of  the  mission,  the  warfighter  must  maintain 
vigilance  and  may  be  required  to  identify  enemy  combatants  obscured 
by  unfamiliar  and  variegated  natural  environment  (e.g.,  mountain  ter¬ 
rain  or  urban  village)  and  may  not  be  easily  distinguished  from  the 
noncombatant  population.  These  tasks  must  also  be  executed  while 
maintaining  a  high  degree  of  situational  awareness  of  other  physical 
and  cognitive  demands. 

Virtual  Technology  and  the  Military 

Over  the  last  20  years,  technological  advances  in  almost  every 
aspect  of  combat  performance  have  altered  the  way  the  warfighter 
prepares,  approaches,  and  completes  combat-related  tasks.  Current 
warfare  is  a  multifaceted  engagement,  with  the  warfighter  serving  as 
the  most  complex  component  of  the  weapon  system.  Researchers  con¬ 
tinually  investigate  the  cognitive  and  physical  stresses  of  the  warf¬ 
ighter  and  propose  actions  to  improve  performance  and  mitigate 
injuries.  Traditionally,  human  performance  research  has  struggled  to 
balance  testing  conditions  that  emulate  the  real  world  with  the  control 
and  precision  of  a  laboratory  setting.  This  becomes  especially  difficult 
in  combat-related  environments. 

In  response,  military  leadership  has  guided  the  creation  of  realistic, 
operationally  relevant  training  environments  outside  of  theater,  such 
as  the  Marine  Corps  Infantry  Immersive  Trainers.  Because  field  testing 
of  combat  operations  is  generally  unfeasible  or  at  least  logistically 
challenging,  immersive  environments  provide  a  fitting  alternative  for 
scientists  to  study  the  warfighter.  To  further  bring  the  field  into  the 
laboratory,  virtual  technology  has  emerged  as  a  leading  tool  to  define 
the  experiences  of  warfighters  during  combat-type  operations  (1). 

The  military  has  a  long  history  of  using  virtual  reality  for  training. 
As  early  as  1910,  the  aviation  community  operated  simplistic  ground- 
based  flight  simulators  to  train  pilots,  with  the  Link  Flight  Trainer 
(Link  Aviation  Devices  Inc.,  Binghamton,  NY),  becoming  one  of  the 
most  widely  used  during  World  War  II  (3,6).  Driving  simulators  like 
the  Common  Driver  Trainer  (Science  Applications  International 
Corporation,  McLean,  VA)  are  currently  used  to  train  warfighters  to 
maneuver  military  vehicles  in  unfamiliar  terrain  (e.g.,  mine-resistant 
ambush-protected  vehicles  through  Afghanistan)  without  damage  to 
vehicles  or  personnel.  As  technology  advanced,  these  simulators,  like 
the  Comet  IV  simulator  (Redifon,  Crawley,  UK)  included  haptic  (force) 
feedback  on  the  controls  and  platforms  that  produced  pitch,  yaw,  and 
roll  motions.  However,  simulation  of  infantry-type  tasks,  when  the 
warfighter  has  "boots  on  the  ground,"  dramatically  increases  the  com¬ 
plexity  of  an  immersive  environment.  During  human  movement,  the 
brain  subtly  registers  visual,  auditory,  and  tactile  inputs  regarding  the 
position  of  the  person  as  he  or  she  progresses.  These  subtle  cues  are 
difficult  to  replicate  in  a  controlled  laboratory  setting.  Fortunately,  the 
exponential  growth  of  technology  in  the  last  decade  has  evolved  vir¬ 
tual  reality  and  immersive  applications  for  medicine  and  research. 
Visual  display  synchronization  and  hardware  control  innovations. 


largely  from  the  commercial  video-gaming  industry,  have  made  im¬ 
mersion  in  virtual  environments  extremely  realistic  to  the  user. 

The  Warfighter  Performance  Laboratory  at  the  Naval  Health  Re¬ 
search  Center  (NHRC)  in  San  Diego,  CA,  has  installed  a  state-of-the- 
art  tool  called  the  Physical  and  Cognitive  Operational  Research 
Environment  (PhyCORE)  that  has  as  its  base  a  Computer  Assisted  Reha¬ 
bilitation  Environment  (CAREN)  Extended  System  (Motek  Medical  Inc., 
Amsterdam,  Netherlands;  http://www.motekmedical.com/products/ 
caren/).  The  PhyCORE  is  a  unique  tool  that  provides  an  immersive 
and  interactive  experience  in  which  the  subject  responds  in  real  time 
to  visual,  auditory,  vestibular,  and  tactile  inputs  (Fig.  1).  While  tradi¬ 
tional  testing  equipment  is  limited  to  the  examination  of  individual 
aspects  of  human  performance  (usually  cognitive  or  physical  only), 
the  PhyCORE  engages  multiple  sensory  systems  and  has  versatile 
utility  for  diagnostics,  rehabilitation,  and  research. 

System  Capabilities 

Currently,  there  are  four  different  CAREN  systems  within  the 
United  States  Department  of  Defense  (NHRC,  Walter  Reed  National 
Military  Medical  Center,  Center  for  the  Intrepid,  and  National  Intrepid 
Center  of  Excellence)  and  approximately  20  others  in  the  world.  The 
systems  are  primarily  used  for  rehabilitation  of  patients  with  complex 
injuries.  At  NHRC,  we  have  expanded  the  capabilities  to  meet  the 
technical  and  operational  demands  of  addressing  clinical  and  perfor¬ 
mance  research  questions.  Using  the  PhyCORE,  researchers  measure 
and  analyze  balance,  posture,  gait,  motor  learning,  vigilance,  response 
time,  and  decision  making.  Hardware,  including  a  synchronized  scent 
system  (ScentAir,  Charlotte,  NC),  wireless  controllers,  and  weighted 
airsoft  weapons,  have  been  added  to  improve  the  overall  immersion 
experience.  Wireless  electromyography,  accelerometers,  EKG,  and 
force  transducers  (Delsys  Inc.,  Boston,  MA)  have  also  been  incorpo¬ 
rated  for  further  data  acquisition  and  novel  control  mechanisms. 

By  integrating  virtual  reality  principles  with  traditional  testing 
constraints,  the  PhyCORE  provides  a  realistic  environment  for  con¬ 
trolled  and  repeatable  experiments.  The  PhyCORE  occupies  approxi¬ 
mately  2,000  ft2  of  the  Warfighter  Performance  Lab  located  at  the 
Naval  Third  Fleet  Complex  in  San  Diego,  C  A.  A  majority  of  the  system 
resides  within  a  cylindrical  concrete  pit  (5  ft  deep  X  14.5  ft  in  diame¬ 
ter)  in  the  laboratory  floor.  A  circular  platform  (9  ft  diameter)  is  situ¬ 
ated  atop  a  motion  base  secured  to  the  foundation  of  the  pit.  Four 
electrical  actuators  linearly  move  this  platform  up  to  16"  in  the  fore, 
aft,  and  lateral  directions,  up  to  7"  in  the  vertical  direction,  and  up  to 
20°  around  each  rotational  axis  (pitch,  yaw,  and  roll).  A  dual-belt  (side 
by  side)  treadmill  with  instrumented  force  plates  are  integrated  into 
the  moveable  platform.  The  pit  is  surrounded  (180°)  by  a  9  ft  tall 
curved  screen  which  displays  the  environment  generated  from  three 
projectors.  This  graphical  environment  is  synchronized  with  the  move¬ 
ment  of  the  platform  and  treadmill.  Surrounding  the  system  are  12 
optical  motion  capture  cameras  (Motion  Analysis  Corporation,  Santa 
Rosa,  CA).  Retroreflective  markers  can  be  placed  on  anatomical  land¬ 
marks  and/ or  objects  (e.g.,  a  handheld  weapon).  Traditionally,  motion 
capture  is  used  to  collect  positional  data  of  body  segments  and  joint 
rotations  during  normal  movement.  In  the  PhyCORE,  the  motion  cap¬ 
ture  system  and  CAREN  D-flow  software  further  process  positional 
data  points  for  real-time  system  interaction. 
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Fig.  1.  The  Physical  and  Cognitive  Operational  Research  Environ¬ 
ment  (PhyCORE),  which  provides  an  interactive,  immersive  environment 
for  use  in  diagnostics,  rehabilitation,  and  research. 


The  CAREN  system  comes  standard  with  several  basic  applica¬ 
tions  that  can  be  used  for  a  variety  of  patient  populations,  but  custom 
applications  can  also  be  created  for  specific  purposes  or  interventions. 
One  of  the  more  popular  standard  programs,  the  "Boat"  scenario,  re¬ 
quires  the  user  to  pilot  a  virtual  boat  through  a  marine  slalom  course. 
This  course  consists  of  numbered  buoys,  virtual  sharks,  and  land- 
masses  that  must  be  avoided  and  properly  circumnavigated.  During 
the  course,  the  platform  moves  in  synch  with  the  motion  of  the  boat  on 
the  waves.  The  interaction  can  be  varied  depending  on  user  ability  or 
familiarization  level.  Alterations  in  maximum  boat  speed  (movement 
through  the  scene),  turning  sensitivity  (platform  rotations),  wave 
height  (vertical  platform  displacement),  wave  speed  (platform  rota¬ 
tion/displacement  frequency),  and  obstacles  are  adjusted  before  or 
during  the  course  by  the  system  operator.  This  scenario  has  been  used 
at  NHRC  as  part  of  a  research  program  to  measure  learning  ability  of 
able-bodied  subjects  in  this  environment,  and  for  a  group  of  patients 
using  this  as  part  of  a  vestibular  rehabilitation  program. 

Additional  programs  include  "Endless  Road"  and  "City  Street" 
scenarios,  where  users  walk  on  the  treadmill  through  a  rural  outdoor 
path  or  a  typical  city  street,  respectively.  We  have  used  the  Endless 
Road  scene  as  part  of  our  vestibular  rehabilitation  program  in  trau¬ 
matic  brain-injured  patients.  At  random  intervals  and  locations,  sim¬ 
ple  math  problems  or  other  cognitive  tasks  are  displayed  to  force 
patients  to  rotate  their  head  and  scan  their  visual  field  while  walking. 
In  the  City  Street  program,  the  goal  is  to  avoid  common  obstacles, 
such  as  street  signs,  moving  vehicles,  and  pedestrians.  The  complexity 
of  the  scene  can  range  from  a  simple  sidewalk,  a  few  buildings,  and  a 
few  pedestrians  to  a  complex  urban  atmosphere  with  large  skyscrap¬ 
ers,  crowds  of  pedestrians,  and  city  traffic. 

Using  each  of  these  commercial  rehabilitation  programs  as  models, 
we  developed  a  mountain  terrain  (virtual  Afghanistan)  similar  to  the 
environmental  experience  of  recent  conflicts.  The  program  is  synchro¬ 
nized  with  the  treadmill  and  motion  platform  to  reflect  changes  in  ter¬ 
rain  (incline,  decline,  slant)  and  immerse  the  warfighter.  We  have 
added  operational  objects  to  the  program,  such  as  military  vehicles,  air 
support,  equipment,  and  hazards  (e.g.,  improvised  explosive  devices). 
As  the  warfighter  progresses  through  the  environment,  combatants 
and  noncombatants  appear  in  the  landscape  and  the  warfighter  must 
appropriately  target  and  shoot  the  combatant  using  the  integrated 
weapons  system.  This  program  is  similar  to  commonly  used  commer¬ 
cial  first-person  shooting  games.  Currently,  this  program  is  used  as 
part  of  our  vestibular  rehabilitation  study  and  it  will  also  be  used  to 
examine  the  combined  physical  (e.g.,  heavy  carriage  loads,  hypoxia) 


and  cognitive  stresses  (decision  making,  memory)  experienced  by  in¬ 
fantry  warfighters. 

Research 

We  are  establishing  the  reference  and  standards  for  movement, 
learning,  and  adaptations  within  the  immersive  system.  Previous  re¬ 
search  has  shown  spatiotemporal  differences  in  the  way  people  move 
over  ground  vs.  on  a  traditional  treadmill  (2,4,5).  The  lack  of  proprio¬ 
ceptive  and  visual  cues  on  a  treadmill  cause  subtle  differences  in  the 
response  of  the  individual  (e.g.,  slower  preferred  walking  speeds,  al¬ 
tered  foot-ground  contact).  These  differences  become  relevant  when 
attempting  to  explore  the  effects  of  diverse  physical  and  cognitive 
military  protocols.  The  PhyCORE  addresses  this  by  providing  realistic 
optic  flow  and  feedback  synchronized  with  the  movement  of  the 
treadmill  and  motion  platform. 

We  are  currently  examining  the  spatiotemporal  characteristics  of 
normal  walking  in  the  PhyCORE.  We  hypothesize  that  immersive  en¬ 
vironments  will  significantly  reduce  the  differences  observed  during 
walking  on  a  traditional  treadmill  compared  with  overground  terrain. 
Further,  we  have  developed  realistic  scenarios  to  reflect  current  opera¬ 
tional  missions.  With  these,  we  can  measure  cognitive  performance 
with  tasks  such  as  memory  recall,  reaction  time,  target/ no  target,  and 
navigation.  While  operating  in  this  scenario,  the  warfighter  can  be  out¬ 
fitted  with  the  current  standard  tactical  gear,  uniform,  and  weighted 
mock  weapons.  This  provides  the  ability  to  address  difficult  opera¬ 
tional  research  questions  by  monitoring  physiological,  cognitive,  and 
performance  metrics  difficult  to  collect  in  field  settings. 

To  prepare  and  protect  the  modern  warfighter  requires  under¬ 
standing  three  facets  of  human  research:  how  do  we  reset  the  injured, 
enhance  performance,  and  facilitate  resilience?  Virtual  environment 
technology  will  enable  development  in  all  three  facets  in  both  the 
physical  and  cognitive  domains.  The  current  expansion  of  NHRC's 
PhyCORE  system  has  increased  our  ability  to  analyze  realistic,  opera¬ 
tionally  relevant  tasks  with  the  validity  of  a  research  laboratory 
environment.  Integration  of  video-gaming  technology,  artificial  intel¬ 
ligence,  and  established  measurement  devices  will  expand  the  system 
into  a  complete  warfighter  assessment  tool.  As  we  foster  the  ability  to 
demonstrate  pertinent  warfighter  tasks  in  the  laboratory,  we  will 
enhance  our  ability  to  proactively  prevent  injury  and  increase  the 
strength  of  our  warfighting  community. 
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